FIG. 1 is a schematic diagram illustrating the relationship between the incidence angle δ of an incident ray 109 and the exit angle θ of an exit ray 111 in accordance with a projection scheme which is useful in an imaging optical system such as a camera. The camera comprises a lens 110 and a camera body 112 having an image sensor therein. The lens 110 is assumed to be rotationally symmetric about an optical axis 101. In an ideal camera model, the lens is considered as a pinhole. The nodal point N of the lens is the point corresponding to the position of the pinhole. The incident ray 109 which is originated from a point 123 of an object (hereinafter, referred to as an object point) enters into the lens and then the imaging lens forms an image point P on an image plane 132 which is a portion of the focal plane of the lens. The focal plane is a plane perpendicular to the optical axis 101. In order to obtain a sharp image in good focus, the image sensor plane 114 of the camera image sensor should be coincident with the focal plane, and the image plane is a sub-region of the focal plane wherein the actual image points have been formed from the exit rays. Hereinafter, it will be assumed that the focal plane is always coincident with the image sensor plane.
If the distance d from the lens to the object point is sufficiently large, then the incident ray 109 may be considered as a parallel beam, and the distance from the nodal point N of the lens to the focal plane is substantially identical to the effective focal length f of the lens. The whole bundle of exit rays which pass through the lens and form image points on the image plane are considered to pass through the nodal point N of the lens 110 in the camera. For simplicity of notation, the ray 109 before passing through the nodal point is referred to as the incident ray, while the ray 111 after passing through the nodal point is referred to as the exit ray. For a rotationally symmetric lens, the object point 123, the incident ray 109, the nodal point N, the exit ray 111, the image point P and the optical axis 101 are all contained in a same plane. Such a plane is referred to as the incidence plane.
When an imaging system is said to be distortion-free, then the lens employed in the imaging system follows a rectilinear projection scheme. In a rectilinear projection scheme, an object point is considered to lie on a plane (hereinafter, referred to as an object plane 121). The object plane is perpendicular to the optical axis 101, and the distance from the nodal point N of the camera to the object plane measured along the optical axis is d, and the lens 110 causes the image of the object point 123 to be formed as an image point P on the focal plane. Both the object plane 121 and the focal plane 132 are planes which are perpendicular to the optical axis.
For simplicity of notation, a rectangular coordinate system having the optical axis 101 as the z-axis is used, and the coordinate origin coincides with the nodal point N. In that case, the rectangular coordinate of an object point is given as (x′, y′, d), while the rectangular coordinate of an image point is given as (x, y, −f). Further, the distance r (hereinafter, referred to as an image height) from the intersection point O between the image plane (i.e., the focal plane) and the optical axis 101 to the image point P on which the exit ray 111 is captured is given by Equation 1 as follows:r=√{square root over (x2+y2)}  MathFigure 1
If the nadir angle of the exit ray with respect to the optical axis 101 of the camera is referred to as θ, then the image height r is given by Equation 2 as follows:r=f tan θ  MathFigure 2
In the present invention, the zenith angle is an angle measured from the positive z-axis toward the negative z-axis, while the nadir angle is an angle measured from the negative z-axis toward the positive z-axis. For example, the zenith angle of the incident ray 109 is δ, while the nadir angle thereof is 180°−δ. Further, the nadir angle of the exit ray 111 is θ while the zenith angle thereof is 180°−θ. By definition, the sum of the zenith angle and the nadir angle is 180°. The incidence angle or the exit angle of the present invention may be measured by the zenith angle or by the nadir angle, and whether it is measured by the zenith angle or the nadir angle must be clear from the accompanying drawings.
The distance L (hereinafter, referred to as an object height) from the optical axis to the object point is given by Equation 3 as follows:L=√{square root over ((x′)2+(y′)2)}{square root over ((x′)2+(y′)2)}  MathFigure 3
If the incidence angle of the incident ray 109 with respect to the optical axis 101 is δ, then the incidence angle δ and the object height L satisfies the relation given in Equation 4 as follows:L=d tan δ  MathFigure 4
In a conventional rectilinear projection scheme, the tangent of the incidence angle of the incident ray is proportional to that of the exit angle of the exit ray as in Equation 5,tan θ=crl tan δ  MathFigure 5
where crl is proportionality constant. Such a projection scheme has well been known for a long time and is described in detail in References 1 and 2. If the incidence angle of an incident ray for such an optical system ranges from δ=0 to δ=δ2 and the exit angle of an exit ray ranges from θ=0 to θ=θ2, then the following relationship is obtained from Equation 5.tan θ2=crl tan δ2  MathFigure 6
The proportionality constant crl can be uniquely determined from Equation 6, and therefore the incidence angle δ of an incident ray for a rectilinear projection scheme is given as a function of the exit angle θ of an exit ray as in Equation 7.
                              δ          ⁡                      (            θ            )                          =                              tan                          -              1                                ⁡                      [                                                            tan                  ⁢                                                                          ⁢                                      δ                    2                                                                    tan                  ⁢                                                                          ⁢                                      θ                    2                                                              ⁢              tan              ⁢                                                          ⁢              θ                        ]                                              MathFigure        ⁢                                  ⁢        7            
If a lens implementing the rectilinear projection scheme as described above is used, then the image of the object on the object plane 121 is faithfully reproduced on the focal plane. The image height r divided by the object height L is referred to as a magnification M of the lens. The magnification is given by Equation 8 from Equations 2 and 4 as follows:
                    M        =                              r            L                    =                                                    f                ⁢                                                                  ⁢                tan                ⁢                                                                  ⁢                θ                                            d                ⁢                                                                  ⁢                tan                ⁢                                                                  ⁢                δ                                      =                                          c                                  r                  ⁢                                                                          ⁢                  1                                            ⁡                              (                                  f                  d                                )                                                                        MathFigure        ⁢                                  ⁢        8            
Since crl is a constant, the magnification depends only on the object distance d and the focal length f of the lens, and therefore, the respective portions of the object on the object plane 121 are reproduced on the image sensor plane 114 with their correct ratios maintained. That is, the image captured by a lens implementing a rectilinear projection scheme is an undistorted image. The rectilinear projection scheme is a projection scheme which is satisfied or at least aimed at to be satisfied by most lenses and considered as the most natural projection scheme.
The incidence angle δ of an incident ray in a rectilinear projection scheme cannot be larger than 90°. However, the incidence angle δ of an incident ray in some application needs to be equal to or even larger than 90°, and therefore, projection schemes other than the rectilinear projection scheme are required in such a case. An equidistance projection scheme is the most widely known projection scheme among these projection schemes. In the equidistance projection scheme, it is assumed that an object point exists on a large spherical surface 131 surrounding the camera, so that the incidence angle δ of an incident ray and the tangent of the exit angle θ of an exit ray is proportional to each other as in Equation 9,tan θ=cedδ  MathFigure 9
where ced is another proportionality constant. Considering the range of the incidence angle and the range of the exit angle as described in the rectilinear projection scheme, the incidence angle is given as a function of the exit angle as in Equation 10.
                              δ          ⁡                      (            θ            )                          =                              (                                          δ                2                                            tan                ⁢                                                                  ⁢                                  θ                  2                                                      )                    ⁢          tan          ⁢                                          ⁢          θ                                    MathFigure        ⁢                                  ⁢        10            
Although the exit angle θ in an equidistance projection scheme cannot be not larger than 90°, there is no limit to the incidence angle δ of an incident ray. Accordingly, the object point 133 on the spherical surface 131 from which the incident ray 109 is originated may lie even behind the camera. A high-end fisheye lens implements the equidistance projection scheme in a relatively faithful manner, and therefore, how small an error is between the equidistance projection scheme and the projection scheme of the actual lens is used as an important index for measuring the performance of the fisheye lens.
In a stereographic projection scheme, which is similar to an equidistance projection scheme, the incidence angle δ of an incident ray and the exit angle θ of an exit ray satisfies the relationship as given in Equation 11.
                              tan          ⁢                                          ⁢          θ                =                              c            sg                    ⁢          tan          ⁢                      δ            2                                              MathFigure        ⁢                                  ⁢        11            
Considering the range of the incidence angle and the range of the exit angle, the incidence angle is given as a function of the exit angle as in Equation 12.
                              δ          ⁡                      (            θ            )                          =                  2          ⁢                                          ⁢                      tan                          -              1                                ⁢                      ⌈                                                            tan                  ⁢                                                            δ                      2                                        2                                                                    tan                  ⁢                                                                          ⁢                                      θ                    2                                                              ⁢              tan              ⁢                                                          ⁢              θ                        ⌉                                              MathFigure        ⁢                                  ⁢        12            
Among the projection schemes wherein the field of view (FOV) can be equal to or larger than 180° such as those of fisheye lenses, the stereographic projection scheme is considered as the most natural projection scheme. For example, if a camera provided with a fisheye lens implementing a stereographic projection scheme is aimed toward the zenith to take a photograph of the whole sky, a circular object such as the sun always appears as a circle regardless of the image position on the screen. However, in an equidistance projection scheme, a circle may be shown as an ellipse depending on the position on the screen.
The following Equation 13 shows the relationship between the incidence angle δ of an incident ray and the exit angle θ of an exit ray in an orthographic projection scheme.tan θ=cog sin δ  MathFigure 13
Similar to the previous examples, the incidence angle is given as a function of the exit angle as in Equation 14.
                              δ          ⁡                      (            θ            )                          =                              sin                          -              1                                ⁡                      [                                                            sin                  ⁢                                                                          ⁢                                      δ                    2                                                                    tan                  ⁢                                                                          ⁢                                      θ                    2                                                              ⁢              tan              ⁢                                                          ⁢              θ                        ]                                              MathFigure        ⁢                                  ⁢        14            
FIG. 2 is a graph wherein the four different projection schemes used in conventional imaging systems, i.e., a rectilinear projection scheme rl, an equidistance projection scheme ed, a stereographic projection scheme sg and an orthographic projection scheme og are compared to each other. In FIG. 2, the proportionality constants are selected so that the incidence angle is equal to the exit angle (i.e., θ=δ) when they are small. Specifically, they are given as crl=ced=cog=1.0 and csg=2.0. Meanwhile, FIG. 3 is a graph showing the four different projection schemes wherein the common range of the exit angle θ is from 0° to 30° and the common range of the incidence angle δ is from 0° to 60°. Reference 3 provides examples of aspheric lens shapes precisely implementing the projection schemes as described above.
The projection schemes given by Equations 1 through 14 define the ranges of the incidence angle and the exit angle, and the functional relationship between these two variables. However, the effective focal length or equivalently the refractive power is often considered as one of the most important features of a lens. The refractive power is defined by the reciprocal of the effective focal length. The effective focal length of a lens is often the feature which is given the first priority when designing an imaging lens. Accordingly, projection schemes directly reflecting the effective focal length of the lens may be preferred in certain circumstances. Projection schemes directly including the effective focal length are described in Reference 4. According to Reference 4, given the effective focal length f, a rectilinear projection scheme is given by Equation 15 as follows:r=f tan δ  MathFigure 15
Hereinafter, the projection scheme given by Equation 5 is referred to as a first type of rectilinear projection scheme while the projection scheme given by Equation 15 is referred to as a second type of rectilinear projection scheme.
Meanwhile, a second type of stereographic projection scheme is given by Equation 16 as follows:
                    r        =                  2          ⁢                                          ⁢          f          ⁢                                          ⁢          tan          ⁢                      δ            2                                              MathFigure        ⁢                                  ⁢        16            
A second type of equidistance projection scheme is given by Equation 17 as follows:r=fδ  MathFigure 17
Meanwhile, a second type of orthographic projection scheme is given by Equation 18 as follows:
                    r        =                  2          ⁢                                          ⁢          f          ⁢                                          ⁢          sin          ⁢                      δ            2                                              MathFigure        ⁢                                  ⁢        18            
Using Equation 2, Equation 15 may be given as follows:r=f tan θ=f tan δ  MathFigure 19
If f is eliminated from Equation 19, it may be seen that the incidence angle δ is identical to the exit angle θ as in Equation 20.θ=δ  MathFigure 20
Accordingly, if Equation 2 is valid, then the second type of rectilinear projection scheme given by Equation 15 may be considered as a particular example of the first type of rectilinear projection scheme given by Equation 5 (i.e., crl=1.0).
An optical layout of a fisheye lens with the field of view of 180° is given in Reference 4, and the lens design data is given in Reference 5. FIG. 4 is a diagram showing the optical layout and the ray traces for the fisheye lens according to the optical design data given in Reference 5, wherein the effective focal length of the lens is 100.0286 mm. This lens is composed of a first lens element E1 through a ninth lens element E9, a color filter F, and a stop S. The lens is located between an object plane 421 and a focal plane 432. The first lens element E1 is a lens element which an incident ray 409 originating from an object point 423 on the object plane first encounters when it enters into the lens, and the first lens element E1 has a first lens surface R1 which is a refractive surface facing the object plane (i.e., on the object side) and a second lens surface R2 which is another refractive surface facing the image plane (i.e., on the image side). A lens surface according to the present invention is a single surface in a lens element having a refractive power, and may be a refractive surface as described in this example or alternatively a reflective surface. In other words, a lens surface is a concept encompassing a refractive surface and a reflective surface.
Similarly, the second lens element E2 is located directly behind the first lens element, and is a refractive lens element having a third lens surface R3 which is a refractive surface on the object side and a fourth lens surface R4 which is another refractive surface on the image side. In similar manners, each of the other lens elements has two refractive lens surfaces. The filter F is located between the fourth lens element E4 and the stop S and has no refractive power. The incidence angle δ of an incident ray 409 originating from an object point is measured as a zenith angle, wherein the z-axis coincides with the optical axis. Also, the exit angle θ of the corresponding exit ray 411 is measured as a nadir angle, and the image height is designated as rrp.
As described above, most of the fisheye lenses are designed in accordance with an equidistance projection scheme. Shown in FIG. 5 are the ideal image heights red on the focal plane of this fisheye lens determined by the equidistance projection scheme and the effective focal length and the real image height rrp as a function of the incidence angle δ of an incident ray 409. The ideal image height red is given by Equation 17 and drawn as a solid line in FIG. 5. When a complete optical design data is available, the real image height rrp (δ) on the focal plane corresponding to a given incidence angle δ of an incident ray may be calculated using an operator such as ‘REAY’ in professional lens design program ‘Zemax’. The real projection scheme of a lens in the present invention represents a theoretical projection scheme obtained by the method as described above, and has an advantage in that the projection scheme is obtained in a much simpler and more precise way than empirically obtaining from an experiment with a real lens. It can be seen from FIG. 5 that the real image height rp of the lens represented by a dotted line is not substantially different from the ideal image height ed represented by a solid line. FIG. 6 shows the distortion as a percentage error between the ideal image height and the real image height defined by Equation 21.
                              distortion          ⁢                                          ⁢                      (            δ            )                          =                                                                              r                  ed                                ⁡                                  (                  δ                  )                                            -                                                r                  rp                                ⁡                                  (                  δ                  )                                                                                    r                ed                            ⁡                              (                δ                )                                              ×          100          ⁢          %                                    MathFigure        ⁢                                  ⁢        21            
It can be seen from FIG. 6 that this particular fisheye lens is excellent in that the error between the real projection scheme and the ideal projection scheme is less than 4%.
Meanwhile, the projection scheme of this lens may be compared to the first type of equidistance projection scheme represented by Equation 10. However, since the exit angle θ rather than the incidence angle δ is the independent variable, Equation 10 needs to be converted into Equation 22 in which the incidence angle δ is used as the independent variable in order to be compared with Equation 17.
                              tan          ⁢                                          ⁢          θ                =                              (                                          tan                ⁢                                                                  ⁢                                  θ                  2                                                            δ                2                                      )                    ⁢          δ                                    MathFigure        ⁢                                  ⁢        22            
FIG. 7 shows the ideal image height ed from the first type of equidistance projection scheme given by Equation 22 and the real projection scheme rp, and FIG. 8 shows a percentage error given by Equation 23:
                              distortion          ⁢                                          ⁢                      (            δ            )                          =                                                            tan                ⁢                                                                  ⁢                                                      θ                    ed                                    ⁡                                      (                    δ                    )                                                              -                              tan                ⁢                                                                  ⁢                                                      θ                    rp                                    ⁡                                      (                    δ                    )                                                                                      tan              ⁢                                                          ⁢                                                θ                  ed                                ⁡                                  (                  δ                  )                                                              ×          100          ⁢          %                                    MathFigure        ⁢                                  ⁢        23            
It can be seen that the maximum distortion is less than 4% in FIG. 6, while the maximum distortion is more than 8% in FIG. 8. Accordingly, it can be seen that this fisheye lens relatively faithfully implements the second type of equidistance projection scheme given in Equation 17, while it largely differs from the first type of equidistance projection scheme given in Equation 10 or its equivalent Equation 22. As shown above, it can be appreciated that the two equidistance projection schemes respectively given by Equations 10 and 17 are substantially different from each other.
FIG. 9 shows an optical layout and ray traces for a prior art of catadioptric wide-angle lens described in Reference 6. This lens has the field of view of 151.8°, a distortion less than 1%, and an effective focal length f of 0.752 mm. This lens is comprised of seven lens elements, wherein the first lens surface of the first lens element E1 is an aspheric mirror surface, while the second lens element E2 through the seventh lens element E7 are refractive lens elements with both lens surfaces being spherical surfaces. The stop S is located between the first lens element E1 and the second lens element E2, and an optical low pass filter F is located between the seventh lens element E7 and the focal plane 932. The optical low pass filter F is not a constitutional element of the lens but a constitutional element of the camera and is covered over the image sensor plane of the camera image sensor. As previously mentioned, the image sensor plane is considered as coincident with the focal plane. The optical low pass filter serves to remove a moire effect from the image. FIG. 9 indicates that the lens has been designed with a due consideration for the optical low pass filter.
FIG. 10 is a graph showing how faithfully the second type of the rectilinear projection scheme given by Equation 15 is implemented by this catadioptric wide-angle lens, wherein a solid line represents the ideal projection scheme given by Equation 15 and a dotted line represents the projection scheme of a real lens. As shown in FIG. 10, there is no substantial difference between the two projection schemes. FIG. 11 shows the percentage error calculated using Equation 21, and it can be appreciated that the maximum error is less than 1%, and this catadioptric wide-angle lens is practically distortion-free. Meanwhile, as given in Equation 24, the first type of the rectilinear projection scheme given by Equation 7 can be given as a function having the incidence angle as the independent variable.
                              tan          ⁢                                          ⁢          θ                =                                            tan              ⁢                                                          ⁢                              θ                2                                                    tan              ⁢                                                          ⁢                              δ                2                                              ⁢          tan          ⁢                                          ⁢          δ                                    MathFigure        ⁢                                  ⁢        24            
FIG. 12 is a graph comparing the first type of the rectilinear projection scheme rl given by Equation 24 with the real projection scheme rp, and FIG. 13 shows the percentage error calculated in the same manner as in Equation 23. From FIG. 13, it can be seen that the maximum distortion is as much as 70%. Accordingly, although the catadioptric wide-angle lens shown in FIG. 9 faithfully implements the second type of the rectilinear projection scheme given by Equation 15, the error between the real projection scheme and the first type of the rectilinear projection scheme given by Equation 7 is so large that comparing the two projection scheme appears meaningless.
FIG. 14 is a graph clarifying the reason why the difference between the two rectilinear projection schemes is so large that they appear totally unrelated to each other. If the assumption given in Equation 2 is valid, the relation given by Equation 19 should be satisfied. However, from a real projection scheme derived from a lens design data sufficiently in detail for ready manufacturing, it can be seen that f tan δ and f tan θ are vastly different from each other. The relation given in Equation 2 has been considered as a golden rule in the field of geometrical optics and computer science, and the notions of a nodal point and an effective focal length constitute a theoretical background. Since, however, these concepts are valid only within the approximation of paraxial ray tracing, they can be safely applied to an imaging system where the maximum incidence angle is relatively small. Therefore, it is not legitimate to apply these concepts to an ultra wide-angle lens where the maximum incidence angle is relatively large.
In other words, the assumption of Equation 2 is relatively well satisfied by an image formed by a single thin lens element such as a magnifying glass. As a result, the first type of projection scheme and the second type of projection scheme are compatible to each other. However, even in such a case, the assumption of Equation 2 is no more than an approximate relation satisfied within the limit of paraxial ray tracing and it is not a mathematically exact equation. In addition, since the modern advanced optical lens has many lens elements and the field of view is getting ever wider, the assumption of paraxial ray tracing is no more valid. Further, an imaging system using an optoelectronic image sensor such as CCD, CMOS, or the like requires that the exit angles of the exit rays with respect to the optical axis be relatively small. Accordingly, most of the currently designed and manufactured lenses are image-side telecentric lenses, and regardless of the incidence angles, the corresponding exit angles are nearly 0°. Therefore, a lens which does not satisfy Equation 2 is actually preferable, and a pinhole camera is the only imaging system which literally satisfies the Equation 2.
For such a reason, a lens satisfying the first type of projection scheme and a lens satisfying the second type of projection scheme are substantially different from each other, and especially, designing the two types of lenses result in completely different constraints in the merit function. For example, a projection scheme satisfying the Equation 5 defines the range of the incidence angle of the incident ray and the range of the exit angle of the exit ray and the functional relationship between these two variables. Accordingly, if Equation 5 is used as a constraint in a lens design, the effective focal length of the lens becomes a floating variable. Meanwhile, if the Equation 15 is used as a constraint, the effective focal length of the lens becomes a constraint, and according to the assumption of the Equation 2, the maximum incidence angle δ2 of the incident ray is determined by the maximum image height r2 on the image sensor plane. Therefore, as a matter of fact, it can be seen that Equations 5 and 15 correspond to largely different projection schemes. Similar relations also hold in the equidistance projection scheme, in the stereographic projection scheme, and in the orthographic projection scheme.
FIG. 15 is a schematic view of an object plane 1521 used in a panoramic imaging system described in Reference 7. A panoramic imaging system is a system which captures in a single image all the views seeable from a given spot when one turns around 360° at the standing point. Panorama camera which captures beautiful scenery of every direction in a single photograph is one example of a panoramic imaging system. FIG. 16 is a schematic diagram of an imaginary unwarped (i.e., raw) panoramic image plane 1632 obtained by capturing the scene on the object plane 1521 in FIG. 15 with a panoramic imaging system. The views in all directions, i.e., 360°, are captured within a ring-shaped region between an inner rim 1632a and an outer rim 1632b of the unwarped panoramic image plane 1632.
FIG. 17 is a schematic diagram of a catadioptric panoramic imaging system in which the assumption of Equation 2 is premised. A catadioptric panoramic imaging system 1700 having such a function includes as constituent elements a catadioptric panoramic lens having a panoramic mirror surface 1716 and a refractive lens 1710, and a camera body 1712 having an image sensor plane 1714 included therein. The panoramic mirror surface 1716 and the refractive lens 1710 are rotationally symmetric about an optical axis 1701, and the optical axis 1701 coincides with the z axis of the coordinate system. Characteristic of an optical system having an odd number of reflective lens elements, an exit ray 1711 propagates along the negative z axis. In an ideal camera model, the refractive lens 1710 is considered as a pinhole, and the nodal point N of the refractive lens is the point corresponding to the pinhole.
FIG. 18 is an exemplary view of an unwarped panoramic image plane 1832 captured by such a catadioptric panoramic imaging system. When such a panoramic image plane 1832 is unwrapped with respect to the break line 1832c shown in FIG. 18, an unwrapped panoramic image plane 1932 as shown in FIG. 19 is obtained. In order to obtain an unwrapped panoramic image plane shown in FIG. 19 from a raw panoramic image plane shown in FIG. 18, a software image processing or a hardware image processing may be taken.
As schematically shown in FIG. 17, the object plane 1721 is in a form of a cylindrical wall which is rotationally symmetric about the optical axis 1701. As described above, a panoramic imaging system captures the scene on the cylindrically-shaped object plane 1721 around the camera as a ring-shaped image plane on the image sensor plane 1714 which is coincident with the focal plane. An exit ray 1711 originated from an object point 1723 of an object existing on the object plane and propagating toward an image point on the image sensor plane 1714 of the camera is considered as reflected at a point M on the panoramic mirror surface 1716 and then passes through the nodal point N of the refractive lens 1710. The distance from the nodal point to the image sensor plane 1714 generally agrees with the effective focal length f of the refractive lens 1710. For convenience of illustration, the ray 1709 before being reflected at the panoramic mirror surface is referred to as an incident ray, the ray after being reflected therefrom is referred to as a reflected ray, and the reflected ray after passing thorough the nodal point N of the refractive lens is referred to as the exit ray 1711. According to the assumption of Equation 2, the reflected ray is parallel to the exit ray 1711. The incidence angle δ of an incident ray and the exit angle θ of an exit ray in this example are all measured as nadir angles, and the distance r from the center of the image sensor plane, i.e., the intersection point O between the image sensor plane 1714 and the optical axis 1701, to an image point P on the image sensor plane on which the exit ray 1711 is captured is given by Equation 2 as usual.
If a panoramic lens implementing the rectilinear projection scheme is used, the height of an object point 1723 on the object plane 1721, i.e., the distance measured parallel to the optical axis, is proportional to the image height r, i.e., the distance from the center O of the image sensor plane to the corresponding image point P. It is assumed that the axial radius and the height of a point M on the panoramic mirror surf ace 1716 at which the incident ray is reflected are ρ and z, respectively, while the axial radius and the height of the object point 1723 are D and H, respectively. Further, let us assume that the elevation angle of the incident ray 1709 is μ. Herein, the elevation angle is an angle measured from the horizontal plane (i.e., the x-y plane) toward the zenith. Then, the height H of the object point is given by Equation 25 as follows:H=z+(D−ρ)tan μ  MathFigure 25
If the distance from the camera to the object plane is much larger than the size of the camera itself (i.e., D>>ρ, H>>z), Equation 25 may be approximated to Equation 26 as follows:H=D tan μ  MathFigure 26
Accordingly, if the radius D of the object plane is fixed, the height of an object point is proportional to tan μ while the image height is proportional to tan θ. If tan μ is proportional to tan θ as described above, the image of an object on the object plane is captured on the image sensor plane with its respective proportions correctly maintained. Referring to FIGS. 15 and 16, it can be seen that both the elevation angle of the incident ray and the nadir angle of the exit ray have their upper bounds and lower bounds. The elevation angle of the incident ray ranges from μ1 to μ2 (μ1≦μ≦μ2), and the nadir angle of the exit ray ranges from θ1 to θ2 (θ1≦θ≦θ2). Accordingly, in order for the proportionality relationship as described above to be satisfied within the valid ranges of the elevation angle of the incident ray and the nadir angle of the exit ray, the elevation of the incident ray should be given as a function of the exit angle of the exit ray as in Equation 27.
                              μ          ⁡                      (            θ            )                          =                              tan                          -              1                                ⁡                      [                                                                                (                                                                  tan                        ⁢                                                                                                  ⁢                                                  μ                          2                                                                    -                                              tan                        ⁢                                                                                                  ⁢                                                  μ                          1                                                                                      )                                                        (                                                                  tan                        ⁢                                                                                                  ⁢                                                  θ                          2                                                                    -                                              tan                        ⁢                                                                                                  ⁢                                                  θ                          1                                                                                      )                                                  ⁢                                  (                                                            tan                      ⁢                                                                                          ⁢                      θ                                        -                                          tan                      ⁢                                                                                          ⁢                                              θ                        1                                                                              )                                            +                              tan                ⁢                                                                  ⁢                                  μ                  1                                                      ]                                              MathFigure        ⁢                                  ⁢        27            
The shape of a panoramic mirror surface satisfying the rectilinear projection scheme given by Equation 27 is given by Equations 27 through 29 and described in detail in References 8 and 9,
                              R          ⁡                      (            θ            )                          =                              R            ⁡                          (                              θ                1                            )                                ⁢                      exp            ⁡                          [                                                ∫                                      θ                    1                                    θ                                ⁢                                                                                                    sin                        ⁢                                                                                                  ⁢                                                  θ                          ′                                                                    +                                              cot                        ⁢                                                                                                  ⁢                        ψ                        ⁢                                                                                                  ⁢                        cos                        ⁢                                                                                                  ⁢                                                  θ                          ′                                                                                                                                    cos                        ⁢                                                                                                  ⁢                                                  θ                          ′                                                                    -                                              cot                        ⁢                                                                                                  ⁢                        ψ                        ⁢                                                                                                  ⁢                        sin                        ⁢                                                                                                  ⁢                                                  θ                          ′                                                                                                      ⁢                                                                          ⁢                                      ⅆ                                          θ                      ′                                                                                  ]                                                          MathFigure        ⁢                                  ⁢        28                                ψ        =                              θ            +                          (                                                π                  2                                -                μ                            )                                2                                    MathFigure        ⁢                                  ⁢        29            
where a preferable range of the exit angle may be determined by considering the effective focal length of the refractive lens and the size of the image sensor plane.
FIG. 20 shows a preferable size of an unwarped panoramic image plane 2032 for a given size of an image sensor plane. The image sensor plane is assumed to be the one of a conventional CCD (charge coupled device) or CMOS (complementary metal-oxide-semiconductor) sensor in which the ratio of the horizontal width W to the vertical height H is 4:3. Here in this example, the height H in FIG. 20 is 7.8 mm. Since the goal of a panoramic imaging system is to equally monitor all the directions, i.e., 360°, the diameter of the outer rim 2032b of the unwarped panoramic image plane should not be larger than the height H of vertical side 2014c or 2014d of the image sensor plane 2014 so that the ring-shaped raw panoramic image plane is completely contained within the image sensor plane. Further, if the circumference of the inner rim 2032a is vastly different from that of the outer rim 2032b, the image resolutions in the upper and the lower portions of the unwrapped panoramic image plane will be greatly different from each other. Accordingly, it is preferable that the ratio of the circumferences or equivalently the ratio of the radii is about 1:2. Considering the matters as described above, it is a sensible choice that the radius of the outer rim of the unwarped panoramic image plane is the half of the vertical height of the image sensor plane, and the radius of the inner rim is again half of the radius of the outer rim.
FIG. 21 is a schematic diagram showing an exit ray 2111 with the maximum exit angle. If a panoramic image plane as shown in FIG. 20 is assumed, the exit ray having the maximum exit angle is a ray arriving at the horizontal side 2114a of the image sensor plane. If the effective focal length of the refractive lens is f, the size of the maximum exit angle is given by Equation 30 as follows:
                              θ          v                =                              tan                          -              1                                ⁡                      (                          H                              2                ⁢                                                                  ⁢                f                                      )                                              MathFigure        ⁢                                  ⁢        30            
Since it is very difficult to design a satisfactory refractive lens when the maximum exit angle is too large (i.e., when the field of view is too large), the maximum exit angle should have an appropriate value, which in turn means that the effective focal length of the refractive lens should have an appropriately large value.
FIG. 22 is a preliminary optical layout in which the effective focal length of the refractive lens and an approximate mirror size are selected considering the matters as described above. The nodal point is divided into a front nodal point F.N. and a rear nodal point R.N. under the assumption that the refractive lens is a thick lens. In this case, the distance from the rear nodal point to the image sensor plane approximately corresponds to the effective focal length f of the refractive lens. Since it is difficult to design a satisfactory refractive lens when the maximum exit angle of the exit ray is too large, the effective focal length f is chosen to be 8 mm, and accordingly, the maximum exit angle is given as 25.989°. Meanwhile, the distance zo from the front nodal point to the lowest point on the mirror surface 2216 is set as 40 mm, and the maximum axial radius ρ2 of the mirror surface is approximately 27.5 mm. When obtaining the mirror surface profile, the elevation angle of the incident ray has been ranged from −60° to +30°. FIG. 23 shows the mirror surface profile as determined by the aforementioned design variables and the paths of chief rays.
In order to complete the design of a catadioptric rectilinear panoramic lens as describe above, a dedicated refractive lens which is to be used with the panoramic mirror 2316 shown in FIG. 23 should be designed. To design a dedicated refractive lens, a merit function should be written. Although the elevation angle of the incident ray has been ranged from −60° to +30° when designing the rectilinear panoramic mirror itself, the rectilinear panoramic lens as a whole is designed for the elevation angle ranging from −30° to +30°, and as a result, the central portion of the rectilinear panoramic mirror surface is not used. Using the range of the exit angle of the exit ray and the relationship of Equation 2, the axial radius of the image point on the image sensor plane ranges from r1=f tan θ1 to r2=f tan θ2 (r≦r≦r2). Accordingly, the projection scheme given by Equation 27 is equivalently given as follows:
                              r          ⁡                      (            μ            )                          =                              r            1                    +                                                                      r                  2                                -                                  r                  1                                                                              tan                  ⁢                                                                          ⁢                                      μ                    2                                                  -                                  tan                  ⁢                                                                          ⁢                                      μ                    1                                                                        ⁢                          (                                                tan                  ⁢                                                                          ⁢                  μ                                -                                  tan                  ⁢                                                                          ⁢                                      μ                    1                                                              )                                                          MathFigure        ⁢                                  ⁢        31            
For the design of a dedicated refractive lens, the functional relationship of Equation 31 is included in the merit function.
FIG. 24 shows the entire optical layout of the rectilinear panoramic lens including the rectilinear panoramic mirror shown in FIG. 23 and the refractive lens along with the ray traces. The panoramic lens includes a first lens element E1, i.e., the rectilinear panoramic mirror with an aspheric surface profile that can be described as a polynomial; a second lens element E2, of which the second lens surface R2 facing the first lens element is described as an even aspheric lens formula and the third lens surface R3 is a spherical refractive surface; and a third lens element E3 through the twelfth lens element E12 sequentially positioned. Each of the third to twelfth lens elements is a refractive lens element with both surfaces being spherical surfaces. The stop S is located between the sixth lens element and the seventh lens element. An incident ray originating from an object point on the object is first reflected at the first lens element, and then sequentially passes through the second to twelfth lens elements, and finally as an exit ray, converges toward an image point on the focal plane.
FIG. 25 is a schematic diagram showing the definition of distortion for a rectilinear panoramic lens, wherein Pa is an image point having the ideal image height rrl and Pb is another image point corresponding to the real projection scheme. The distortion of a rectilinear panoramic lens is defined in a manner similar to Equation 21, and FIG. 26 is a graph showing the distortion calculated using such a scheme and shows that the percentage error (i.e., distortion) is less than 0.2%. Accordingly, the distortion in a prior art of rectilinear panoramic lens described in Reference 7 is practically non-existent.
The rectilinear panoramic lens described in Reference 7 is clearly different from the other panoramic lenses of previous designs in that the lens is substantially distortion-free and the vertical field of view (i.e., the range of the elevation angle of the incident ray) is completely independent of the range of the exit angle of the exit ray. However, the same drawback as those of other wide-angle lenses of prior art persists in that the effective focal length of the refractive lens is used in the projection scheme. Further, the panoramic mirror already satisfies the projection scheme (or the constraint) given by Equations 27 through 29, and the whole panoramic lens including the panoramic mirror satisfies an additional projection scheme given by Equation 31. Since two projection schemes are satisfied, there is a disadvantage in that the lens structure is unnecessarily complicated.    Reference 1: R. Hill, “A lens for whole sky photographs,” Q. J. R. Meteor. Soc. 50, 227-235 (1924).    Reference 2: C. Beck, “Apparatus to photograph the whole sky,” J. Sci. Instr. 2, 135-139 (1925).    Reference 3: Korean Patent No. 10-0624052, “OPTICAL COMPONENTS INCLUDING LENSES HAVING AT LEAST ONE ASPHERICAL REFRACTIVE SURFACE.”    Reference 4: K. Miyamoto, “Fish-eye lens,” J. Opt. Soc. Am. 54, 1060-1061 (1964).    Reference 5: W. J. Smith, Modern Lens Design (McGraw-Hill, Boston, 1992) p. 166.    Reference 6: G. Kweon, S. Hwang-bo, G. Kim, S. Yang, Y. Lee, “Wide-angle catadioptric lens with a rectilinear projection scheme,” Appl. Opt. 45, 8659-8673 (2006).    Reference 7: G. Kweon, M. Laikin, “Design of a mega-pixel grade catadioptric panoramic lens with the rectilinear projection scheme,” J. Opt. Soc. Korea, 10, 67-75 (2006).    Reference 8: Korean Patent No. 10-0491271, “PANORAMIC MIRROR AND IMAGING SYSTEM USING THE SAME.”    Reference 9: Korean Patent No. 10-0552367, “RECTILINEAR MIRROR AND IMAGING SYSTEM HAVING THE SAME.”    Reference 10: N. L. Max, “Computer graphics distortion for IMAX and OMNIMAX projection,” Proc. NICOGRAPH, 137-159 (1983).